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ABSTRACT 


The  military  relies  heavily  on  computer  systems.  Without  a  strong  method  of  authentication 
to  access  these  systems,  threats  to  confidentiality,  integrity,  and  availability  of  government 
information  are  likely  to  be  more  successful.  A  recent  method  of  authentication  for  the 
Windows  8  and  Windows  10  operating  systems  is  picture  gesture  authentication  (PGA), 
a  new  approach  to  entering  a  password  to  authenticate  a  user  during  system  login.  Each 
PGA  password  is  composed  of  three  gestures  that  are  drawn  over  a  picture  chosen  by  the 
user.  Strength  requirements  are  set  for  PGA  passwords  similarly  to  text-based  passwords. 
For  simplicity,  users  tend  to  use  shapes,  colors,  and  objects  in  a  picture,  called  points  of 
interest  (POI),  as  guidance  when  creating  each  gesture  for  their  password.  This  concept 
provides  an  opportunity  for  potential  hackers  to  make  logical  password  guesses,  decreas¬ 
ing  the  security  of  PGA.  Previous  work  on  PGA  security  used  a  proprietary  brute-force 
algorithm  to  guess  passwords  based  on  POIs.  We  present  a  similar  brute-force  algorithm 
that  is  publicly  available.  We  evaluate  the  efficiency  of  the  new  algorithm  against  various 
background  pictures  and  propose  strength  requirements  to  improve  the  security  of  PGA. 
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CHAPTER  1: 

Introduction 


1.1  Motivation 

The  use  of  passwords  as  a  method  of  authenticating  someone’s  claim  of  identity  dates  back 
to  ancient  times  in  the  Roman  military  in  which  Romans  referred  to  passwords  as  “watch¬ 
words”  [1],  Since  then,  passwords  have  been  modified  to  what  we  have  today.  Traditional 
computer-based  authentication  methods  use  text-based  passwords,  which  are  a  string  of 
alphanumeric  characters  and  symbols  used  to  authenticate  a  user  before  granting  that  user 
access  to  a  device  or  program.  For  security  purposes,  many  programs  use  strength  require¬ 
ments  for  passwords.  Strength  requirements  may  include  a  certain  number  of  uppercase 
alphabetic  characters,  lowercase  alphabetic  characters,  symbolic  characters,  or  numerical 
characters,  and  a  minimum  and  maximum  length.  They  may  also  require  a  password  to  be 
changed  after  a  period  of  time,  and  that  no  repeated  passwords  may  be  used.  Even  with 
these  strength  requirements,  there  remain  weaknesses  in  text-based  passwords. 

Suo  et  al.  said  “human  factors  are  often  considered  the  weakest  link  in  a  computer  security 
system”  [2],  Zhao  et  al.  found  that  people  use  simple  passwords  because  they  are  easier 
to  remember  [3],  [4],  Therefore,  dictionary  attacks  were  created,  where  a  list  of  plausible 
passwords  are  generated  based  on  dictionary  words,  and  used  to  guess  passwords.  Other 
human  factors  related  to  text-based  passwords  include  users  recycling  passwords  through¬ 
out  different  programs  or  re-using  passwords  for  the  same  program.  Users  also  tend  to 
write  down  their  password,  either  on  a  sticky  note  left  on  their  desk  or  in  an  unencrypted 
document  on  their  system.  In  either  case,  if  the  password  is  found  there  can  be  numerous 
consequences  to  security.  If  a  password  gets  in  the  wrong  hands,  it  can  lead  to  illicit  access 
onto  a  private  network,  or  a  data  breach. 

Since  text-based  passwords  are  difficult  for  people  to  keep  track  of,  other  methods  of  au¬ 
thentication  have  been  developed.  Suo  et  al.  believe  that  people  are  more  likely  to  remember 
a  visual  password  [2],  Picture  gesture  authentication  (PGA)  is  a  new  type  of  authentication 
that  uses  picture-based  passwords,  and  is  the  scope  of  this  thesis.  In  particular,  the  research 
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looks  at  the  best  types  of  background  pictures  for  more  secure  PGA. 

We  proceed  by  presenting  a  brute-force  algorithm,  designed  after  the  work  of  Zhao  et 
al.  [3],  [4],  that  makes  logical  guesses  to  crack  the  PGA  password  of  a  user  given  a  specific 
picture.  We  programmed  the  algorithm  to  use  points  of  interest  (POI),  which  are  specific 
areas  of  a  picture  that  may  catch  the  eye  of  a  user,  to  determine  likely  choices  of  a  pass¬ 
word.  By  analyzing  the  accuracy  and  efficiency  of  the  algorithm  to  generate  brute-force 
passwords,  we  determine  the  variety  of  pictures  that  are  superior  for  a  background  picture. 
We  show  that  the  background  picture  selected  can  increase  the  strength  of  the  password 
chosen  for  PGA. 


1.2  Benefits  to  the  Navy 

The  main  contribution  of  this  research  is  to  investigate  the  security  bounds  of  picture  ges¬ 
ture  authentication.  The  Navy  would  benefit  from  this  study  because  if  PGA  is  not  a  strong 
method  of  authentication,  then  potential  threats  to  confidentiality,  integrity,  and  availability 
of  government  information  are  plausible.  Strong  authentication  is  recommended  by  the 
DOD  Cybersecurity  Discipline  Implementation  Plan  that  was  amended  February  2016. 


Reducing  anonymity  as  well  as  enforcing  authenticity  and  accountability  for 
actions  on  DOD  information  networks  improves  the  security  posture  of  the 
DOD.  The  connection  between  weak  authentication  and  account  takeover  is 
well-established.  Strong  authentication  helps  prevent  unauthorized  access,  in¬ 
cluding  wide-scale  network  compromise  by  impersonating  privileged  adminis¬ 
trators.  Commanders  and  Supervisors  will  focus  attention  on  protecting  high- 
value  assets,  such  as  servers  and  routers,  and  privileged  system  administrator 
access.  This  line  of  effort  supports  objective  3-4  in  the  DOD  Cyber  Strat¬ 
egy,  requiring  the  DOD  CIO  to  mitigate  known  vulnerabilities  by  the  end  of 
2016.  [6] 


An  agreement  between  Microsoft  and  the  DOD  provides  the  Navy  with  the  newest  versions 
of  Microsoft  products,  including  Windows  8  and  10,  which  both  use  PGA.  Navy  Rear  Ad¬ 
miral  David  G.  Simpson,  DISA’s  vice  director  and  senior  procurement  executive  explained 
that  the  DOD  has  continued  to  focus  on  mobile  computing,  stating  “Microsoft  is  committed 
to  making  sure  that  the  technology  within  the  agreement  has  a  mobile-first  focus,  and  we 
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expect  to  begin  to  take  advantage  of  Microsoft’s  mobile  offerings  as  part  of  our  enterprise 
mobility  ecosystem”  [2],  Microsoft  claims  that  PGA  passwords  are  more  secure  than  text- 
based  passwords  [5],  and  that  DOD  users  will  be  more  likely  to  use  PGA.  It  is  important, 
however,  that  PGA  not  be  used  in  an  insecure  fashion,  therefore,  this  study  is  important  to 
help  the  Navy  make  the  best  decision  on  background  pictures  for  the  security  of  PGA. 


1.3  Thesis  Organization 

The  remainder  of  this  thesis  is  organized  as  follows.  In  Chapter  2,  we  describe  the  history, 
notation,  and  brute-forcing  of  PGA,  as  well  as  related  work.  Chapter  3  discusses  the  two 
corpora  used  to  test  the  program  created  for  this  thesis.  The  process  of  POI  extractions  and 
functions  used  for  the  brute-force  algorithm  are  covered  in  Chapter  4.  In  Chapter  5,  the 
passwords,  POIs,  and  results  of  the  program  for  each  picture  are  explained.  Finally,  the 
conclusions  and  recommendations,  and  suggested  future  work  of  this  thesis  are  presented 
in  Chapter  6. 
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CHAPTER  2: 

Background  and  Related  Work 


In  this  chapter,  we  explain  the  process  of  using  picture  gesture  authentication  (PGA)  on 
a  Windows  8  device.  A  key  insight  is  how  users  tend  to  select  points  of  interest  (POI)  to 
choose  the  location  of  gestures.  POIs  are  a  key  concept  employed  by  prior  work  on  brute¬ 
forcing  a  password  under  PGA.  We  also  summarize  related  work  on  picture  authentication 
schemes.  For  clarity  and  ease  of  comparison,  we  adopt  the  notation  of  Zhao  et  al.  [3],  [4]. 

2.1  Picture  Gesture  Authentication 

Authentication  is  any  mechanism  used  to  validate  if  someone  is  the  identity  they  claim  to 
be  on  a  computer  system  or  program.  There  are  three  broad  approaches  to  authentication, 
often  referred  to  as  something  you  know,  something  you  own,  or  something  you  are.  PGA 
is  a  relatively  new  authentication  mechanism  that  falls  under  the  umbrella  of  something 
you  know.  Microsoft  started  using  PGA  as  an  optional  replacement  for  text  passwords  with 
their  Windows  8  consumer  technology.  This  new  method  of  authentication  was  announced 
by  Microsoft  in  late  2011  [5]  for  all  versions  of  Windows  8  and  products  supporting  PGA 
as  a  primary  method  of  authentication  were  released  on  October  26,  2012. 

With  the  Windows  8  operating  system,  by  default,  user  accounts  are  configured  to  use  text- 
based  passwords.  To  use  PGA,  the  user  selects  the  picture  password  sign-in  option.  After 
providing  proper  credentials,  the  user  is  required  to  choose  a  picture  from  their  picture 
library.  Using  their  own  picture,  instead  of  Microsoft  providing  one,  will  increase  the 
security  of  PGA.  The  intuition  is  that  two  users  are  likely  to  select  different  pictures,  as 
PGA  is  configurable  per-user.  After  a  picture  is  chosen,  the  user  is  prompted  to  create 
a  password.  A  password  for  picture  gesture  authentication  (PGA)  is  a  series  of  gestures, 
limited  to  taps,  circles  or  lines  drawn  on  the  picture.  The  users  are  expected  to  draw  three 
gestures  on  the  picture  using  their  finger  or  stylus  on  the  touchscreen  or  a  mouse  if  no 
touchscreen  is  available.  When  users  later  authenticate  with  PGA,  they  must  redraw  the 
selected  gestures,  in  the  original  order,  on  their  chosen  picture. 

We  record  a  gesture  password  as  a  sequence  of  three  gestures,  n  =  ninjn?,.  Each  n  is 
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one  of  many  passwords  in  the  password  space,  n  e  ]”[•  Each  gesture  in  the  password  is 
represented  as  a  7 -tuple:  7r(-  =  (g,xi,yi,X2,y2,f,d).  Let  g  e  {tap,  circle,  line}  bo  the 
type  of  gesture.  The  first  coordinate  (x\ ,  y\ )  can  indicate  a  tap  point,  the  center  of  a  circle 
or  the  first  point  of  a  line.  The  second  coordinate  ( X2 ,  i/2)  represents  the  end  of  a  line, 
and  is  unused  for  other  gesture  types  (i.e.,  let  (x2,  i/2)  =  (0,0)  for  a  tap  or  circle).  Let 
r  be  the  radius  of  a  circle  gesture,  and  otherwise  unused  (i.e.,  r  =  0  for  a  line  or  tap). 
Let  d  e  {+,  -,0}  be  the  direction  in  which  a  circle  is  drawn,  indicating  a  clockwise  or 
counterclockwise  gesture,  and  otherwise  unused  (i.e.,  0  for  a  tap  or  line).  Each  gesture  is 
one  of  many  possible  gestures  in  the  gesture  space,  7 r,  e  ["[• 


Figure  2.1  shows  an  example  gesture  password.  The  first  gesture,  n\  =  (circle,  35, 15, 0, 0, 9,  -), 
is  a  counterclockwise  circle  around  the  man’s  head  centered  at  (35, 15)  with  a  radius  of  9. 

The  second  gesture,  712  =  (line,  54, 34, 79, 27, 0, 0),  is  a  line  from  (54, 34)  to  (79, 27),  from 
one  woman’s  nose  to  another’s.  The  last  gesture,  773  =  (tap,  16, 35, 0, 0, 0, 0),  is  a  tap  on 
the  left  woman’s  nose,  at  coordinate  point  (16, 35). 


Figure  2.1.  Example  of  a  Sequence  of  Gestures  on  a  Picture.  Adapted 
from  [3],  [4], 


Naturally,  human  error  is  likely  to  occur  when  redrawing  passwords.  Therefore,  a  distance 
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function  is  built  into  the  authentication  process.  Since  pictures  come  in  various  sizes,  the 
longest  dimension  is  divided  into  100  and  the  shortest  dimension  is  scaled  accordingly  [5]. 
The  pictures  are  scaled  to  determine  the  coordinate  points  that  fall  within  an  error  distance 
of  the  actual  coordinate  point  used  for  a  gesture.  When  entering  a  password,  if  a  coordinate 
point  of  a  gesture  is  within  the  error  distance  of  the  actual  coordinate  point,  that  point  will 
be  accepted. 

Figure  2.2  shows  an  example  of  the  points  accepted  during  authentication  within  a  distance 
of  3  around  the  recorded  point  of  the  actual  password  [5].  All  of  the  gesture  points  within 
3  of  the  actual  gesture  point,  shaded  in  green,  are  at  least  90%  accurate  to  the  actual  point 
within  the  error  distance,  and  would  be  accepted  during  user  login.  The  yellow,  orange,  and 
red  points  are  not  close  enough  to  the  actual  gesture  point  to  be  accepted  during  user  login. 
For  example,  a  tap  on  (14, 35)  would  suffice  for  the  gesture  n 3  =  (tap,  16, 35, 0, 0, 0, 0) 
since  the  distance  d((l6, 35),  (14,  35))  <  2V3. 
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Figure  2.2.  Points  <  90%  to  the  100%  Exact  Matched  Point  Are  Accepted 
During  Authentication.  Adapted  from  [5]. 
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2.2  Brute-Forcing  PGA 

Zhao  et  al.  [3],  [4]  provide  the  intuition  that  users  select  gestures  by  employing  points 
of  interest  (POIs)  embedded  in  the  underlying  picture.  POIs  can  be  described  by  many 
features,  such  as  objects  Q>0  =  {head,  eye, mouth, nose, bike,  dog, colors  QJC 
\  hi  nr .  red,  yellow,  green, . . shapes  Q)s  =  {square,  circle,  triangle,  rectangle, . . . } 
and  other  miscellaneous  types,  £Z*.  These  form  an  attribute  space  D  c2®  such  that  = 
%U®cU®sU®*-  Each  POI  is  recorded  as  a  5-tuple,  %  =  (xi,  yi,X2,  y2,  D ),  which 
defines  the  POI  in  picture  k  that  is  enclosed  by  a  rectangle  bounded  by  coordinates  (xi ,  y\  ) 
and  (x2,  yi),  and  has  the  set  of  attributes  rS . 

Referring  back  to  the  working  example  in  Figure  2.1,  the  POIs  include  the  heads  of  each 
person,  their  eyes,  their  noses,  their  mouths,  the  linear  edge  of  the  curtain,  the  blue  lines 
in  the  man’s  shirt,  the  black  dots  on  the  girl’s  shirt,  the  woman’s  necklace,  and  the  corner 
of  the  vanity.  Just  as  users  tend  to  select  dictionary  words  for  text  passwords,  it  is  believed 
that  they  tend  toward  POIs  on  a  picture  to  choose  their  PGA  passwords. 

POIs  help  a  user  remember  where  they  placed  their  gestures.  This  insight  is  used  by  Zhao 
et  al.  to  provide  an  attack  on  PGA,  comparable  to  a  dictionary  attack  against  text-based 
passwords.  As  mentioned  in  Section  2.1,  it  is  unlikely  that  any  two  users  would  use  the 
same  picture  for  authentication.  The  attack  framework  requires  previously  seen  passwords 
on  known  pictures  to  learn  password-selection  patterns  to  create  a  dictionary  of  gesture 
passwords.  Machine  analysis  can  then  be  used  to  identify  POIs  on  pictures  as  a  "dictionary" 
to  guess  a  PGA  password.  This  process  is  discussed  in  more  detail  in  Chapter  3. 


2.3  Related  Work 

There  has  been  growing  interest  in  providing  an  alternative  to  text  passwords  by  using 
graphics.  It  has  been  argued  that  graphical  passwords  are  more  secure  than  text  passwords, 
however,  in  “Graphical  Passwords:  A  Survey,”  Suo  et  al.  explain  how  brute-force  attacks, 
dictionary  attacks,  guessing,  spyware,  shoulder  surfing,  and  social  engineering  are  used 
to  attack  graphical  passwords,  just  like  text  passwords  [2].  They  claim  the  defense  against 
graphical  passwords  is  more  difficult  since  N  length  text  passwords  have  94N  possible  pass¬ 
words  based  on  94  printable  characters.  On  the  other  hand,  PGA  has  only  1,155,509,083 
possible  passwords  with  three  gestures,  based  on  all  the  possible  sets  of  three  gestures  made 
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by  taps,  circles,  and  lines  [5]  whish  is  less  than  946,  (the  number  of  6-character  passwords). 
After  guessing  a  graphical  password,  a  program  must  be  written  to  precisely  draw  such  ges¬ 
tures  on  a  picture.  Suo  et  al.  claimed  in  2005  that  there  was  no  method  of  dictionary  attacks 
on  graphical  passwords.  Since  then,  research  has  shown  that  dictionary  attacks  are  possible 
but  must  be  designed  for  each  individual  picture,  as  described  by  Zhao  et  al.  In  Chapter  4, 
we  explain  how  it  is  easy  to  guess  graphical  passwords  since  they  are  more  predictable 
than  text  passwords.  In  2013,  Damopoulos  et  al.  proved  that  there  exists  a  touchlogger, 
similar  to  a  keylogger  but  for  touch  screens,  that  can  record  gestures  on  touch  screen  de¬ 
vices  [7],  This  is  a  finding  made  after  Suo  et  al.  stated  that  spyware  was  unable  to  track 
picture  passwords.  This  is  important  to  keep  in  mind  since  PGA  is  often  used  on,  though  is 
not  limited  to,  touch  screen  devices.  Picture  passwords  are  vulnerable  to  shoulder  surfing 
as  we  will  discuss  more  in  this  section.  Picture  passwords  are  said  to  be  insusceptible  to 
social  engineering  because  it  is  difficult  to  explain  to  someone  verbally  how  to  recreate  a 
password  [2]. 

One  of  the  vulnerabilities  of  text  passwords  is  that  users  tend  to  recycle  passwords  for 
separate  accounts  because  it  is  difficult  to  remember  multiple  strong  passwords.  Suo  et  al., 
however,  affirm  that  there  is  no  “convincing  evidence”  that  picture  passwords  are  easier 
than  text  passwords  to  memorize.  De  Luca  et  al.  also  conclude  that  authentication  methods 
other  than  text-based  passwords  and  personal  identification  numbers  (PIN)  should  be  used, 
after  analyzing  password  pattern  authentication  [8].  Pattern  passwords  consist  of  a  series 
of  continuous  edges  made  on  a  3x3  grid  of  points.  They  surveyed  users  over  a  period  of 
time  to  collect  data  and  study  the  passwords  the  users  created,  along  with  how  they  created 
them.  Each  user,  they  concluded,  has  a  unique  way  of  making  each  stroke.  If  used  correctly, 
this  pattern  matching  can  be  an  additional  method  of  authentication.  Assuming  an  attacker 
knows  the  shape  of  the  password,  they  may  not  be  able  to  imitate  the  user’s  stroke  motions, 
which  falls  under  the  something  you  are  category  of  authentication. 

Draw  a  Secret  (DAS)  is  a  picture-based  password  authentication  method  that  allows  a  user 
to  make  a  drawing  on  a  blank  grid  as  a  password.  This  is  different  than  PGA  since  there  is 
not  a  background  picture  with  POIs  to  guide  users  in  creating  a  password,  but  is  similar  in 
the  sense  that  a  PGA  password  can  be  a  picture  drawn  on  a  grid  comprised  of  three  gesture 
elements.  Nali  and  Thorpe  prove  this  scheme  is  insecure  by  showing  that  users  center  their 
drawings  and  use  symmetry  [9],  [10],  Essentially,  they  show  that  this  approach  increases 
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the  chances  of  guessing  a  password.  Dunphy  and  Yan  attempted  to  enhance  this  method 
of  authentication  by  providing  a  background  picture  for  a  user  as  a  guide  to  improve  how 
they  originally  created  passwords  [11].  This  scheme  is  called  Background  Draw  a  Secret 
(BDAS).  This  relates  to  PGA  since  they  both  have  a  background  picture  that  directs  users 
in  constructing  a  password,  but  BDAS  has  less  restrictions  on  the  number  and  types  of 
gestures  used  for  creating  a  password.  They  found  that  BDAS  closely  relates  to  PGA  since 
users  are  likely  to  use  POIs.  Since  BDAS  is  like  PGA,  and  PGA  is  insecure,  therefore 
BDAS  is  insecure. 

PassPoints  is  an  authentication  method  that  allows  a  user  to  choose  points  on  a  picture  as 
a  password.  This  is  essentially  a  subset  of  the  password  space  of  PGA,  with  only  the  tap 
gesture  being  allowed.  PassPoints  is  similar  to  DAS,  containing  a  less  structured  pass¬ 
word  space  to  PGA,  but  when  selecting  passwords  PGA  has  fewer  rules  than  PassPoints. 
Wiedenbeck  et  al.  studied  the  security  of  PassPoints  and  found  that  users  tend  to  use  taps 
corresponding  to  POIs,  which  they  call  “hotspots,”  when  choosing  points  that  correspond 
to  POIs.  The  main  outcome  of  their  work  is  the  recommendation  that  users  should  select 
pictures  that  avoid  hotspots  [12],  [13]. 

Wiedenbeck  et  al.  also  found  that  users  rely  on  POIs  to  assist  in  building  passwords.  Using 
the  same  dataset  as  Zhao  et  al.  shown  in  Section  3.1,  Alshehri  et  al.  explored  security  of 
PGA,  restricted  to  pictures  with  a  high  number  of  POIs.  Since  POIs  are  used  to  brute-force 
PGA,  a  background  picture  with  more  POIs  would  represent  a  larger  password  space,  and 
thus  provide  more  security  against  brute-force.  As  yet  unpublished,  they  are  developing 
a  metric  to  find  if  a  picture  is  suitably  complex  by  validating  pictures  with  more  POIs  to 
be  less  resistant  to  dictionary  attacks  [14].  Pictures  with  few  POIs  are  more  susceptible  to 
attacks.  Thus,  Alshehri  et  al.  claim  there  should  be  strength  requirements  of  the  background 
picture.  In  contrast,  we  are  concerned  with  revalidating  the  premises  and  results  of  the 
original  study  by  Zhao  et  al. 

Most  PGA  methods  are  used  with  touch  screen  devices.  In  addition  to  click  points,  as 
mentioned  by  Alshehri  et  al.,  Aviv  et  al.  found  that  smudge  marks  can  be  used  to  guess  the 
passwords  of  any  of  the  aforementioned  types  of  picture  authentication  [15], 

Picture  password  mechanisms  are  also  susceptible  to  shoulder  surfing.  Logging  in  with 
PGA  allows  someone  close  by  to  easily  see  a  user’s  password.  To  provide  more  security, 
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a  system  such  as  LatentGesture  can  help  keep  the  PGA  password  more  secure  [16].  La- 
tentGesture  records  a  user’s  behavior  on  a  touchscreen  device  such  as  the  speed  of  swiping 
across  the  screen  or  typing  patterns.  These  recorded  behaviours  build  a  model  of  that  user. 
When  it  suspects  the  current  user  does  not  match  the  model,  LatentGesture  will  automat¬ 
ically  log  off  the  system.  Saravana  described  a  study  using  20  people  that  were  asked  to 
check  boxes,  swipe  sliding  bars,  and  tap  buttons  to  fill  out  a  form.  With  high  accuracy, 
LatentGesture  was  able  to  identify  the  users  correctly  [17].  This  is  not  a  surprising  result 
because  LatentGesture  combines  the  something  you  know  authentication  category  with  the 
something  you  are  category. 

Overall,  picture  gesture  authentication  has  its  weaknesses  and  vulnerabilities  just  like  text- 
based  passwords.  Thus,  we  created  a  brute-force  algorithm  described  in  Chapter  4  to  com¬ 
pare  the  security  of  one  picture  to  another,  determining  the  best  selection  of  background 
pictures  for  an  increase  in  security  for  PGA.  Before  describing  the  algorithm,  we  will  dis¬ 
cuss  the  data  given  by  Zhao  et  al.  that  we  have  also  used  in  our  study. 
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CHAPTER  3: 
Corpora 


In  this  chapter,  we  discuss  the  data  gathered  and  analyzed  by  Zhao  et  al.  in  their  study  [4], 
Two  corpora  were  employed  in  their  study,  both  containing  pictures  and  passwords  created 
by  the  study’s  subjects.  The  Arizona-Turk  dataset  was  an  artificial  dataset,  where  subjects 
generated  passwords  for  a  small  set  of  images.  The  Arizona-Student  dataset  was  a  more 
authentic  dataset,  where  university  students  generated  personal  passwords  used  by  a  web¬ 
site.  The  next  two  sections  summarize  the  demographics  of  the  subjects  and  the  contents 
of  the  corpora  in  the  study. 


3.1  Arizona-T\irk  Dataset 

The  Arizona-Turk  dataset  (called  dataset  2  in  the  Zhao  et  al.  study  [3],  [4])  was  solicited  by 
advertisements  in  the  schools  of  engineering  and  business  at  two  different  universities,  and 
gathered  using  Amazon’s  Mechanical  Turk  crowdsourcing  service.  Only  individuals  with 
previous  security-related  research  experience  were  qualified  to  participate  so  they  could 
understand  the  importance  of  this  study. 

In  the  Arizona-Turk  dataset,  762  subjects  were  given  15  pictures  (see  Figure  3.1)  drawn 
from  the  PASCAL  Visual  Object  Classes  Challenge  2007  dataset  [18].  The  subjects  were 
prompted  to  pretend  the  pictures  were  protecting  their  bank  information,  with  the  intention 
of  influencing  subjects  to  make  strong  passwords  for  each  of  the  15  pictures.  Not  all  sub¬ 
jects  completed  the  entire  task,  so  the  number  of  passwords  gathered  for  each  picture  is 
not  the  same  (see  Figure  3.2).  A  total  of  10,039  passwords  were  gathered:  on  average,  669 
passwords  per  picture  and  13  passwords  per  subject.  Interestingly,  there  were  passwords 
which  one  might  guess,  such  as  circling  tires  on  a  bike  and  tapping  a  person’s  nose.  Further 
discussion  can  be  found  in  Chapter  4. 

The  subjects  were  given  a  demographic  survey.  Of  the  762  subjects,  only  652  (85.5%) 
filled  out  the  survey.  Of  the  652  surveyed,  420  (64.4%)  of  them  reported  being  male,  232 
(35.6%)  female;  243  (37.2%)  were  between  18  and  24  years  of  age,  296  (45.4%)  between 
25  and  34  years  of  age,  and  98  (15%)  between  35  and  50  years  of  age. 
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(a)  000243.jpg  (b)  000316.jpg  (c)  001116.jpg  (d)  001358.jpg  (e)  002057.jpg 


(f)  002080.jpg  (g)  002840.jpg  (h)  003026.jpg  (i)  003731.jpg  (j)  004054.jpg 


(k)  005570.jpg  (I)  006412.jpg  (m)  006467.jpg  (n)  007628.jpg  (o)  009899.jpg 

Figure  3.1.  The  15  Pictures  from  the  Arizona-Turk  Dataset.  Source:  [3],  [4], 
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Figure  3.2.  Number  of  Passwords  for  Each  of  the  15  Pictures  in  the  Arizona- 
Turk  Dataset 
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As  part  of  the  survey,  a  multiple  choice  question  was  asked  to  help  understand  the  choice 
of  passwords  made  by  the  subjects  as  follows:  “Which  of  the  following  best  describes 
what  you  are  considering  when  you  choose  locations  to  perform  gestures?”  Of  the  subjects 
in  this  study,  389  (59.6%)  answered,  “I  try  to  find  locations  where  special  objects  are”; 
143  (21.9%)  answered,  “I  try  to  find  locations  where  some  special  shapes  are”;  57  (8.7%) 
answered,  “I  try  to  find  locations  where  colors  are  different  from  their  surroundings”;  and 
66  (10.1%)  answered,  “I  randomly  choose  a  location  to  draw  without  thinking  about  the 
background  picture.”  Thus,  90.2%  of  respondents  admitted  to  using  a  strategy  of  selecting 
POIs,  which  effectively  limited  the  password  space  and,  perhaps,  biased  it  toward  a  POI 
populated  area  of  the  picture. 

3.2  Arizona-Student  Dataset 

The  Arizona-Student  dataset  (called  dataset  1  in  the  Zhao  et  al.  study)  was  gathered  from 
university  students  in  a  classroom  setting.  An  authentication  method  modeled  after  PGA 
in  Windows  8  was  created  to  gather  information  on  how  students  in  an  undergraduate  com¬ 
puter  science  class  would  create  passwords.  This  authentication  method  was  used  by  the 
students  to  access  the  course  website,  containing  class  materials  such  as  homework,  assign¬ 
ments,  grades,  and  lecture  notes.  Data  was  gathered  over  one  semester,  or  approximately 
three  and  a  half  months. 

The  publicly  released  dataset  contains  subject  IDs,  a  hash  value  for  the  picture,  a  password, 
and  an  activity  log.  The  log  recorded  setting  of  passwords,  attempted  logins,  the  number 
of  successful  login  attempts,  and  any  password  changes  or  new  picture  selections.  Since 
students  selected  their  own  pictures,  some  contained  family  photos  and  other  personally 
identifiable  information  (PII),  so  no  pictures  were  released  with  the  dataset. 

A  total  of  56  students  in  the  computer  science  class  participated  in  the  study.  The  data  col¬ 
lected  reflected:  69  different  pictures,1  86  unique  passwords,  2,536  login  attempts  (2,109 
successful,  427  failed)  and  172  registrations  (86  registered,  86  confirmations).  On  average, 
each  student  used  2.5  pictures,  made  37.66  successful  login  attempts,  had  7.625  failed  login 
attempts,  registered  1.53  logins,  and  confirmed  1.52  logins  (see  Figure  3.3).  Between  the 

According  to  Zhao  et  al.  [3],  [4],  there  were  58  unique  pictures;  this  does  not  match  the  calculations 
made  with  the  public-released  data. 
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registrations,  confirmations,  and  successful  and  failed  logins,  there  were  a  total  of  2,708 
datapoints. 


Successful  and  Failed  Login  Attempts  per  Subject 


Figure  3.3.  Number  of  Successful/Failed  Login  Attempts  and  Number  of 
Reset  Passwords  per  Subject  in  the  Arizona-Student  Dataset 

The  students  were  also  asked  the  same  demographic  survey.  Of  the  56  students,  only  33 
(58.9%)  filled  out  the  survey.  Of  the  33  surveyed,  27  (81.8%)  reported  being  male,  and  6 
(18.2%)  female;  21  (63.6%)  were  between  18  and  24  years  of  age.  Since  the  students  were 
in  an  undergraduate  course  in  computer  science,  it  is  reasonable  that  the  numbers  were  not 
as  diverse  as  those  for  dataset  2. 

As  in  the  Arizona-Turk  dataset,  the  subjects  of  the  Arizona-Student  dataset  were  also  asked 
the  question,  “Which  of  the  following  best  describes  what  you  are  considering  when  you 
choose  locations  to  perform  gestures?”  Of  the  33  respondents,  24  (72.7%)  answered,  “I  try 
to  find  locations  where  special  objects  are”;  8  (24.2%)  answered,  “I  try  to  find  locations 
where  some  special  shapes  are”;  0  (0%)  answered,  “I  try  to  find  locations  where  colors  are 
different  from  their  surroundings”;  and  1  (3%)  answered,  “I  randomly  choose  a  location 
to  draw  without  thinking  about  the  background  picture.”  Since  students  were  asked  to  use 
this  password  to  protect  their  actual  course  material,  and  to  select  their  own  pictures,  we 
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expect  that  this  dataset  was  more  realistic  than  the  Arizona-Turk  dataset.  This  reflects  an 
even  stronger  trend  toward  biased  password  selection. 
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CHAPTER  4: 
BestCover  Algorithm 


This  chapter  presents  the  BestCover  algorithm  described  by  Zhao  et  al.  [4]  to  make  a 
logical  guess  of  an  unknown  password  using  a  previously  unseen  picture.  In  Section  4. 1 ,  we 
detail  the  POIs  in  the  Arizona-Student  dataset  and  the  Arizona-Turk  dataset.  In  Section,  4.2 
we  define  location  dependent  gesture  selection  functions  and  how  they  are  used  to  map 
POIs  into  potential  passwords  for  a  picture.  In  Section  4.3,  we  then  explain  the  BestCover 
algorithm,  which  uses  a  subset  of  the  dataset  for  training  and  is  evaluated  on  the  remainder 
of  the  dataset.  This  is  the  same  methodology  employed  by  Zhao  et  al.  to  evaluate  this 
algorithm  [3],  [4],  We  adopt  the  notation  of  Zhao  et  al.  for  ease  of  comparison  between  our 
independent  re-implementation  and  their  original  work. 

4.1  POI  Extraction 

For  each  of  the  datasets  in  the  Arizona  case  study,  Zhao  et  al.  extracted  POIs  with  “mature 
computer  vision  techniques  such  as  object  detection,  feature  detection  and  objectness  mea¬ 
sure”  [3],  [4],  The  POI  attributes  were  categorized  as  follows:  face,  body,  eye,  ear,  mouth, 
nose,  head/shoulder,  clock,  airplane,  unknown  object,  forehead,  car,  line  type,  circle  type, 
color  type,  “no  semantics”  and  “not  valid.” 

The  number  of  POIs  extracted  from  the  pictures  in  the  Arizona-Student  dataset  are  ex¬ 
pressed  in  Figure  4.1.  The  number  of  POIs  per  picture  varied  widely  between  the  pictures 
the  students  chose.  Recall  that  for  this  dataset,  some  pictures  were  not  made  available  due 
to  PII  concerns,  however  Zhao  et  al.  [3],  [4]  provided  information  about  the  POIs  (their 
type  and  their  coordinate  location  on  the  picture).  This  eliminated  the  need  to  extract  POIs 
using  computer  vision  methods,  and  thus  reduced  many  variables  in  the  attempt  to  recreate 
an  algorithm  simliar  to  that  of  Zhao  et  al.  to  decide  which  background  pictures  are  best  to 
use  in  PGA.  For  the  Arizona-Turk  dataset,  Figure  4.2  shows  the  number  of  POIs  extracted 
for  each  of  the  15  pictures  in  Figure  3.1.  We  observed  a  correlation  between  the  variation 
in  the  number  of  POIs  per  picture,  and  the  level  of  difficulty  to  brute  force  PGA  passwords, 
described  further  in  Chapter  5. 
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Number  of  POIs  Per  Picture 


Picture 


Figure  4.1.  Number  of  POIs  Extracted  from  the  58  Pictures  in  the  Arizona- 
Student  Dataset 

4.2  Location  Dependent  Gesture  Selection  Functions 

Users  are  likely  to  choose  POIs  on  a  picture  when  selecting  a  password.  Therefore,  map¬ 
pings  were  created  to  aid  the  brute  force  method  described  in  Section  4.3.  Location  depen¬ 
dent  gesture  selection  functions  (LdGSF)  [3],  [4]  are  mappings  s  :  Gx  2^  x  2^  x  0  — >  2^ 
from  descriptions  of  gestures  on  POIs  to  PGA  passwords  using  actual  coordinate  points  of 
those  POIs  in  the  picture.  The  domain  is  the  cross  product  of  the  set  of  gestures,  the  set  of 
attributes  at  the  first  point,  the  set  of  attributes  at  the  second  point  if  the  gesture  is  a  line, 
and  the  set  of  POIs  in  the  given  picture,  respectively.  The  range  is  the  password  space. 
Using  the  POIs  extracted  from  the  picture,  as  described  in  Section  4.1,  a  mapping  can  be 
made  to  describe  gestures  on  a  picture.  A  sequence  of  three  LdGSF  mappings,  T  =  .V1.v2.v3, 
will  yield  three  gestures,  making  plausible  passwords. 

For  example,  referring  to  Figure  2.1  with  the  password  n  =  n  \  7^3 ,  where  n\  = 
(circle,  35, 15, 0, 0, 9,  -),  712  =  (line,  54,  34, 79, 27, 0, 0),  and^  =  (tap,  16, 35, 0, 0, 0, 0), 
the  LdGSFs  for  the  kth  picture  pk  would  be:  si  =  s(circle,  {head},®, 6k),  £2 
slime,  {nose},  {nose}, 6k),  S3  =  s(tap,  { nose },  0, 6k). 
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Number  of  POIs  Per  Picture:  Method  2.1 


Picture 

Figure  4.2.  Number  of  POIs  Extracted  from  the  15  Pictures  in  the  Arizona- 
Turk  Dataset 

An  LdGSF  sequence  can  map  to  several  passwords.  For  example,  given  the  gesture  made 
by  .s  i  =  sicircle,  {head},  0,6k)  above,  if  a  user  decides  to  perform  this  gesture  on  Fig¬ 
ure  2.1,  there  are  four  possible  heads  to  circle  and  each  circle  can  have  a  different  circum¬ 
ference.  Therefore,  one  LdGSF  can  produce  many  possible  gestures,  and  a  single  LdGSF 
sequence  can  produce  many  possible  passwords. 


4.3  Brute-Force  Algorithm 

Since  POIs  on  a  picture  may  decrease  a  user’s  password  space  by  steering  them  toward  spe¬ 
cific  gestures,  a  brute  force  algorithm  centered  around  this  notion  will  assist  in  attacking 
a  password  for  a  previously  unseen  picture.  Zhao  et  al.  describe  the  BestCover  algorithm 
to  create  a  LdGSF  sequence  dictionary.  The  program  written  for  this  algorithm  was  not 
released  to  the  public.  Hence,  we  attempted  to  recreate  their  algorithm  using  known  pass¬ 
words  to  derive  patterns  of  data  that  were  used  to  prioritize  guesses,  providing  the  most 
efficient  coverage  of  the  password  space,  i.e.,  guesses  were  ordered  by  popularity  of  the 
relationship  between  POI  and  gesture.  Figure  4.3,  expresses  in  pseudocode  the  BestCover 
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algorithm  in  a  way  that  aligns  with  our  implementation  of  the  original  work  of  Zhao  et  al. 

1:  function  BestCover((si  , . . . ,  s*n),  (;fi , . . . ,  7?n)) 

2:  for  Si  in  (s“i , . . . ,  s*n)  do 

3:  for  7Tj  in  (^i , ,  j?n)  do 

4:  if  7fj  e  Si  then 

5:  Si  count  +  + 

6:  end  if 

7:  end  for 

8:  end  for 

9:  for  Si  in  (4 , . . . ,  s*n)  do 

10:  if  Si  count  4-  0  then 

11:  S'  {5/  :  s}  count} 

12:  end  if 

13:  end  for 

14:  order  sort  S'  by  max  s}  count 

15:  return  order 

16:  end  function 

Figure  4.3.  The  Pseudocode  of  BestCover .  Adapted  from  [3],  [4], 

First,  the  LdGSF  sequences  were  created  separately.  Each  set  of  attributes  collected  for  the 
LdGSFs  was  built  from  known  passwords.  Since  the  passwords  contain  coordinate  points, 
if  a  point  fell  within  an  interval  of  a  POI’s  location  then  that  attribute  and  its  gesture  were 
recorded  as  an  LdGSF.  If  the  coordinate  point  fell  within  an  intersection  of  multiple  POIs 
then  multiple  attributes  were  added  in  the  LdGSF. 

The  input  to  the  BestCover  algorithm  consists  of  the  training  data’s  LdGSF  sequences  and 
their  corresponding  passwords.  Lines  2-5  verify  the  number  of  passwords  that  the  LdGSF 
sequences  produce  from  the  training  data,  assigning  them  each  a  rating.  The  LdGSF  se¬ 
quences  not  found  to  produce  any  of  the  passwords  are  not  beneficial  to  the  final  dictionary 
to  produce  passwords.  In  lines  9-11,  only  the  LdGSF  sequences  with  a  ranking  greater 
than  zero  are  taken  into  consideration  in  the  dictionary.  After  zero-rank  LdGSF  sequences 
are  removed,  the  remaining  are  ordered  by  rank  in  line  14.  The  highest  ranked  LdGSF 
sequence  is  assigned  the  highest  priority  because  it  is  viewed  as  most  likely  to  generate  a 
correct  password  based  on  its  frequency  in  the  test  data.  The  ordered  list  is  then  returned 
and  used  to  generate  a  password  dictionary. 

To  build  the  password  dictionary,  we  defined  the  CreateDictionary  algorithm  in  Fig- 
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ure  4.4  with  the  main  focus  being  on  the  sets  of  attributes  in  each  LdGSF.  If  even  one 
element  of  the  list  matches  a  POI  in  the  given  picture,  then  the  LdGSF  is  beneficial.  Oth¬ 
erwise,  the  entire  LdGSF  sequence  is  disregarded.  With  a  valid  sequence,  a  search  for  all 
POI  combinations  that  match  the  sequence  attributes  are  found.  The  list  of  combinations 
are  heuristically  ordered  by  pattern  as  described  in  line  5. 

Each  PGA  password  combination  is  described  as  positively  horizontal  if  the  gestures  placed 
in  the  POI  locations  appear  to  be  in  a  left-to-right  order,  negatively  horizontal  for  a  right- 
to-left  orientation,  positively  vertical  if  the  gestures  are  bottom-to-top,  negatively  vertical  if 
top-to-bottom,  and  diagonal  if  they  have  both  a  vertical  and  horizontal  pattern.  According 
to  Zhao  et  al.,  user  gesture  patterns  are  found  to  be  most  common  in  the  following  order: 
positively  horizontal,  diagonal,  positively  vertical,  negatively  horizontal,  negatively  verti¬ 
cal,  and  the  rest  follow.  These  results  could  not  be  reproduced  in  our  work  therefore,  the 
order  of  password  guesses  made  by  Create  Diet  ionary  differed  from  those  in  Zhao  et  al. 
password  dictionary.  This  heuristically  ordered  list  of  applicable  sequences  derived  from 
LdGSFs  is  the  final  password  dictionary.  The  results  collected  on  the  number  of  password 
guesses  may  vary  based  on  the  assumptions  made  in  designing  CreateDictionary. 

1 :  function  CreateDictionary(o rder,  0 0 
2:  for  {s! ,  s~2, 53}  in  order  do 

3:  for  cr\, o"2, (T3  e  Ok  do 

4:  if  crjes}  then 

5:  POIlist  <—  order  by  Horiz+,  Diag,  Vert+,  Horiz-,  Vert-  then  Other 

6:  end  if 

7:  end  for 

8:  end  for 

9:  for  set  €  POIlist  do 

10:  dictionary  <—  (x,-,  yi)  e  set  Vi 

11:  end  for 

12:  return  dictionary 

13:  end  function 

Figure  4.4.  Ordered  LdGSFs  from  Figure  4.3  and  an  Unseen  Picture  are 
Used  to  Brute  Force  a  Password 

Finally,  given  the  algorithms  and  the  data  on  each  picture,  we  were  able  to  generate  pass¬ 
word  guesses  and  keep  count  of  how  many  guesses  were  made  before  each  password  was 
cracked.  These  results  are  analyzed  in  Chapter  5. 
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CHAPTER  5: 
Analysis 


In  this  chapter,  we  analyze  the  POIs  for  the  pictures  in  the  Arizona- Turk  dataset  and  discuss 
the  results  of  the  algorithms  described  in  Chapter  4. 

5.1  Analyzing  Points  of  Interest 

For  this  research,  we  analyzed  the  POIs  of  the  15  pictures  from  the  Arizona-Turk  study, 
shown  in  Figure  3.1.  Due  to  PII  reasons,  we  did  not  have  access  to  the  pictures  in  the 
Arizona- Student  study,  so  we  could  not  analyze  these.  Figure  5.1,  shows  red  dashed  rect¬ 
angles  on  each  of  the  pictures,  representing  extracted  POIs  from  the  images  as  discussed 
in  Section  4.1.  The  type  of  POI  is  labeled  above  each  rectangle.  Each  POI  is  identified 
as  a  face,  a  body,  an  eye,  an  ear,  a  mouth,  a  nose,  a  set  of  head  and  shoulders,  a  clock, 
an  airplane,  a  forehead,  or  a  car.  Some  POIs  were  identified  only  as  line,  circle,  or  color 
type.  Other  POIs  were  identified  as  an  unknown  objects,  or  as  having  no  semantics.  The 
algorithm  only  used  the  previously  listed  POIs  when  creating  passwords. 

The  following  are  the  POIs  that  were  identified  for  each  corresponding  picture  in  Fig¬ 
ure  5.1: 

•  Figure  5.1(a)  is  simply  a  picture  of  an  airplane  in  the  sky,  but  the  POIs  identified  are 
a  nose,  a  mouth,  and  another  POI  with  no  semantics. 

•  Figure  5.1(b)  is  also  an  airplane  in  the  sky,  yet  the  POIs  identified  are  two  eyes  and  a 
POI  with  no  semantics. 

•  Figure  5. 1(c)  is  a  person  with  the  following  identified  POIs:  a  body,  a  face,  three  eyes, 
three  mouths,  three  noses,  4  circle  types,  a  color  type  and  2  POIs  with  no  semantics. 

•  Figure  5.1(d)  is  a  picture  of  children  playing  together  with  the  following  POIs  iden¬ 
tified:  1  body,  6  mouths,  2  eyes,  6  circle  types,  4  color  types,  and  one  with  no 
semantics. 

•  Figure  5.1(e)  is  the  front  of  a  BMW  automobile.  The  POIs  recognized  are  a  clock,  a 
nose,  5  color  types,  3  circle  types,  and  a  POI  with  no  semantics. 

•  Figure  5. 1(f)  is  a  close-up  picture  of  a  train.  The  POIs  identified  are  2  bodies,  7  circle 
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types,  6  color  types,  and  a  POI  with  no  semantics. 

•  Figure  5.1(g)  is  a  car  with  the  following  POIs  identified:  a  face,  a  mouth,  2  noses,  2 
eyes,  4  circle  types,  7  color  types,  and  2  POIs  with  no  semantics. 

•  Figure  5.1(h)  appears  to  be  two  tourists  standing  together.  The  POIs  identified  in  this 
picture  are  2  bodies,  3  mouths,  2  eyes,  5  circle  types,  4  color  types,  a  POI  with  no 
semantics. 

•  Figure  5.1(i)  is  a  picture  of  a  man  and  a  woman.  The  POIs  are  6  mouths,  6  eyes,  2 
faces,  5  circle  types,  and  a  set  of  head  and  shoulders. 

•  Figure  5.1(j)  is  a  picture  with  a  group  of  people.  The  POIs  identified  are  an  eye,  5 
bodies,  7  faces,  and  6  mouths. 

•  Figure  5.1(k)  is  another  picture  of  two  people  with  the  following  identified  POIs:  a 
body,  a  nose,  6  mouths,  2  faces,  3  eyes,  and  6  circle  types. 

•  Figure  5.1(1)  is  a  bicycle  with  the  following  POIs  identified:  a  body,  a  face,  an  eye,  6 
mouths,  6  circle  types,  and  3  POIs  with  no  semantics. 

•  Figure  5.1(m)  is  also  a  bicycle  with  the  following  POIs  identified:  a  body,  a  face,  an 
eye,  4  mouths,  4  circle  types,  2  color  types,  and  2  with  no  semantics. 

•  Figure  5.1(n)  is  a  picture  of  a  man.  The  POIs  identified  are  a  body,  2  noses,  3  eyes,  4 
mouths,  3  circle  types,  a  color  type,  a  set  of  head  and  shoulders,  and  4  POIs  with  no 
semantics. 

•  Figure  5.1(o)  is  a  picture  of  a  dog.  The  POIs  found  are  a  nose,  4  eyes,  2  mouths,  3 
circle  types,  4  color  types,  and  2  with  no  semantics. 

Clearly,  many  POIs  were  incorrectly  identified,  therefore  the  source  of  POI  extraction  ap¬ 
pears  not  to  have  been  well  developed.  This  led  to  major  consequences  when  using  the 
BestCover  algorithm,  which  is  discussed  further  in  Section  5.2. 

Figure  5.2  shows  the  15  pictures  from  the  study  with  their  corresponding  POI  boxes  in 
red  and  associated  passwords  in  blue.  Of  note,  the  password  coordinate  points  tend  to  fall 
within  the  red  POI  boxes.  Specific  shapes  were  used  to  guide  gestures  that  were  made  for 
the  passwords,  for  example  heads  and  wheels  were  circled,  edges  had  lines  associated  with 
them,  and  eyes  were  tapped.  Any  password  guess  with  a  single  gesture  outside  the  scope 
of  the  picture’s  POIs  was  not  cracked.  The  algorithm  made  password  guesses  based  only 
on  information  known  about  the  POIs.  We  did  not  make  password  guesses  outside  the  POI 
boxes  shown  in  red.  We  did,  however,  consider  circles  around  POIs,  as  long  as  their  center 
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(a)  000243  (b)  000316  (c)  001116  (d)  001358  (e)  002057 


(k)  005570  (I)  006412  (m)  006467  (n)  007628  (o)  009899 

Figure  5.1.  Identified  POIs  of  the  15  Pictures  from  the  Arizona-Turk  Dataset 


point  was  in  a  POL  For  example,  Figures  5.2a  and  5.2b  are  pictures  of  a  small  airplane 
with  a  clear  sky  in  the  background  and  with  the  airplane  being  the  only  POI  in  each  picture, 
there  were  many  passwords  with  gestures  made  outside  of  the  POI,  i.e.,  in  the  middle  of 
the  sky.  Table  5.1  represents  the  percentage  of  passwords  with  either  one,  two,  or  all  three 
of  their  gestures  made  within  POIs,  and  indicates  the  chances  of  the  algorithm  cracking  a 
password. 

This  data  showed  how  often  users  rely  on  POIs  in  creating  their  passwords.  For  example, 
in  Figure  5.3,  by  looking  only  at  the  passwords  for  each  of  these  pictures  without  the 
background  pictures  themselves,  it  is  clear  that  the  pictures  are  bicycles. 


5.2  Analyzing  BestCover  Results 

Implementing  the  BestCover  algorithm  (see  Section  4.3)  on  the  Arizona-Turk  dataset  pro¬ 
vided  the  results  shown  in  Figures  5.4  through  5.16.  These  graphs  only  show  data  from 
passwords  that  were  cracked.  The  rest  of  the  passwords  could  not  be  cracked  by  the  al¬ 
gorithm,  therefore,  the  password  guess  count  is  irrelevant.Our  results  were  not  comparable 
to  Zhao  et  al.  since  their  experiments  used  both  the  Arizona-Turk  dataset  and  the  Arizona- 
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Table  5.1.  Percentage  of  Passwords  Possible  to  Guess  with  Number  of  Ges¬ 
tures  in  POIs 


Figure 

%  passwords  with 
all  three  gestures 
outside  of  POIs 

%  had  exactly  two 
gestures  outside 
of  the  POIs 

%  had  only  one 
gesture  outside 
of  the  POIs 

%  passwords 
were  guessable 
using  algorithm 

5.2(a) 

19 

14 

15 

52 

5.2(b) 

20 

19 

15 

46 

5.2(c) 

3 

2 

13 

82 

5.2(d) 

1 

2 

12 

82 

5.2(e) 

7 

13 

16 

86 

5.2(f) 

4 

6 

18 

72 

5.2(g) 

4 

6 

17 

73 

5.2(h) 

6 

8 

21 

65 

5.2(i) 

0 

0 

3 

97 

5.20) 

4 

5 

14 

76 

5.2(k) 

3 

1 

1 

85 

5.2(1) 

0 

0 

6 

94 

5.2(m) 

3 

3 

12 

82 

5.2(n) 

0 

0 

1 

99 

5.2(o) 

5 

6 

19 

70 

Student  datasets. 

Figure  5.4  shows  the  results  of  Figure  3.1(a).  As  mentioned  in  Section  5.1,  there  are  very 
few  POIs  in  this  picture,  and  they  were  not  correctly  identified.  This  made  it  unlikely 
that  the  algorithm  would  crack  the  password  on  this  type  of  picture.  Less  than  30%  of 
the  passwords  were  cracked,  and  the  uncracked  passwords  were  those  with  gestures  found 
outside  of  the  POIs.  The  POIs  took  up  a  small  area  of  this  picture  allowing  the  algorithm  to 
run  quickly.  A  picture  with  a  minimal  amount  of  POIs  should  not  be  used  as  a  background 
choice. 

Figure  5.5  shows  the  results  of  Figure  3.1(b).  Similar  to  the  last  picture,  there  were  very 
few  POIs  in  this  picture,  and  yet  they  were  all  incorrectly  identified.  Due  to  the  lack  of 
POIs,  the  algorithm  only  took  a  few  minutes  to  run,  but  only  cracked  about  30%  of  the 
passwords  due  most  gestures  being  made  outside  of  POIs.  Since  this  picture  did  not  have 
many  POIs,  it  is  not  the  best  choice  for  a  background. 

Figure  5.6  shows  the  results  of  Figure  3.1(c).  There  were  several  POIs,  most  of  which  were 
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(a)  000243 


(b)  000316 


(c)  001116 


(d)  001358 


(e)  002057 


(f)  002080 


(g)  002840 


(h)  003026 


(i)  003731 


(j)  004054 


(k)  005570 


(I)  006412 


(m)  006467 


(n)  007628 


(o)  009899 


Figure  5.2.  Passwords  of  the  15  Pictures  from  the  Arizona-Turk  Dataset 


Figure  5.3.  Passwords  for  Two  Pictures  of  the  Arizona-Turk  Dataset 


accurately  identified,  and  covered  most  of  the  area  of  the  picture,  allowing  the  algorithm 
to  crack  about  40%  of  the  passwords.  Observing  the  results,  we  notice  that  the  majority 
of  the  passwords  were  cracked  within  the  same  range  of  guesses.  This  allows  us  to  think 
of  improvements  for  the  algorithm.  Details  for  improving  the  algorithm  can  be  found  in 
Section  6.2.  Despite  the  higher  password-cracking  rate  of  this  picture,  this  picture  is  a 
better  background  choice  compared  to  the  previous  ones  since  it  has  more  POIs,  but  we 
will  discuss  how  some  of  the  other  pictures  are  superior  choices. 

Figure  5.7  shows  the  results  of  Figure  3.1(d).  The  algorithm  was  able  to  crack  over  30% 
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Figure  5.4.  CDF  Results  of  Picture  000243.jpg 


Figure  5.5.  CDF  Results  of  Picture  000316.jpg 


of  the  passwords.  Observing  the  results,  we  notice  that  about  15%  of  the  passwords  were 
cracked  within  the  same  range  of  guesses.  Assuming  this  jump  on  the  graph  was  made 
from  the  blowup  wheel  in  the  picture,  the  password  guesses  could  have  been  made  sooner 
with  improvements  in  the  algorithm  described  in  Section  6.2.  There  were  unidentified  POIs 
in  this  background  picture  that  were  used  as  guidance  for  gestures.  Since  those  POIs  were 
not  identified,  the  algorithm  was  unable  to  crack  those  passwords. 

Figure  5.8  shows  the  results  of  Figure  3.1(e).  The  algorithm  cracked  over  30%  of  the  pass¬ 
words.  We  were  unable  to  determine  why  so  many  guesses  were  made  before  passwords 
were  cracked.  It  is  believed  that  the  overlap  caused  repeated  guesses  that  should  be  im- 


30 


1.0 


0.8 


i5  0.6 

fU 

o 


o 


0.2 


0.0  ^ - - * - ‘ - ‘ - ‘ - ‘ - 

0  1  2  3  4  5  6 

Number  of  Password  Guesses  le7 

Figure  5.6.  CDF  Results  of  Picture  001116.jpg 


Figure  5.7.  CDF  Results  of  Picture  001358.jpg 


proved  in  the  algorithm.  With  the  POIs  covering  only  half  of  the  picture  and  some  POIs 
not  identified,  this  was  a  stronger  picture  background. 

Figure  5.9  shows  the  results  of  Figure  3.1(f).  The  algorithm  cracked  about  35%  of  the 
passwords.  About  2/3  of  the  passwords  cracked  were  within  the  same  range  of  guesses. 
It  is  safe  to  assume  these  passwords  that  were  cracked  were  the  three  wheels  on  the  train. 
This  picture  is  a  perfect  example  to  explain  how  to  improve  the  algorithm  to  make  guesses 
starting  with  coordinate  points  in  the  midpoint  of  the  POI,  instead  of  bottom-left  to  the 
top-right  as  the  algorithm  works.  More  details  can  be  found  in  Section  6.2.  If  the  wheels 
were  not  the  main  focus  of  users,  this  would  make  a  stronger  background  picture. 
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Figure  5.8.  CDF  Results  of  Picture  002057.jpg 


Figure  5.9.  CDF  Results  of  Picture  002080.jpg 


Figure  5.10  shows  the  results  of  Figure  3.1(g).  The  algorithm  cracked  over  30%  of  the 
passwords.  Observing  the  results,  we  notice  that  about  20%  of  the  passwords  were  quickly 
cracked.  With  the  entire  car  identified  as  a  POI,  these  POIs  were  able  to  be  cracked. 

Figure  5.11  shows  the  results  of  Figure  3.1(h).  The  algorithm  cracked  about  25%  of  the 
passwords.  Observing  the  results,  we  notice  that  about  10-15%  of  the  passwords  cracked 
were  from  circling  the  heads.  Besides  those  passwords,  it  was  very  difficult  to  crack  other 
passwords  with  this  background  since  there  is  so  much  activity  in  this  picture.  This  is  a 
great  example  of  a  secure  background  picture. 

Figure  5.12  shows  the  results  of  Figure  3.1(i).  The  algorithm  cracked  over  30%  of  the 
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Figure  5.10.  CDF  Results  of  Picture  002840.jpg 


Figure  5.11.  CDF  Results  of  Picture  003026.jpg 


passwords  with  25%  of  them  immediately  guessed.  A  close  up  picture  gives  less  interesting 
POIs  of  interest,  making  it  easy  to  guess  the  passwords.  Circling  heads,  tapping  eyes,  and 
connecting  eyes  are  the  first  guesses  made.  Otherwise,  there  were  not  many  passwords 
cracked. 

Figure  5.13  shows  the  results  of  Figure  3.1(j).  The  algorithm  cracked  about  35%  of  the 
passwords  with  25%  of  them  being  a  combination  of  circling  heads  the  heads.  If  users 
were  using  more  of  a  variety  of  POIs,  then  there  would  be  significantly  fewer  passwords 
cracked. 

Figure  5.14  shows  the  results  of  Figure  3.1(m).  The  algorithm  cracked  just  under  30%  of 
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Figure  5.12.  CDF  Results  of  Picture  003731.jpg 


Figure  5.13.  CDF  Results  of  Picture  004054.jpg 


the  passwords.  The  first  15%  of  the  passwords  were  using  the  tires  as  POIs.  Otherwise,  the 
other  passwords  were  difficult  to  crack.  This  is  a  decent  background  picture  since  there  are 
many  POIs  that  can  be  of  interest. 

Figure  5.15  shows  the  results  of  Figure  3.1(n).  The  algorithm  cracked  about  40%  of  the 
passwords.  The  first  15%  were  immediately  identified.  They  must  have  been  in  the  same 
class  of  LdGSFs.  With  the  man’s  face  being  the  main  focus  of  passwords  chosen  by  users, 
this  is  not  the  best  choice  of  a  background  picture. 

Figure  5.16  shows  the  results  of  Figure  3.1(o).  The  algorithm  cracked  about  35%  of  the 
passwords.  About  25%  of  these  passwords  were  guessed  almost  simultaneously.  Altering 
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Figure  5.14.  CDF  Results  of  Picture  006467.jpg 


Figure  5.15.  CDF  Results  of  Picture  007628.jpg 


the  program  to  guess  these  passwords  first  would  be  a  major  improvement.  Not  many  of 
the  other  passwords  were  cracked.  There  was  not  enough  of  a  variety  of  POIs  in  this  picture 
for  users  to  vary  their  passwords,  making  it  a  weak  background  picture. 

Depending  on  the  picture  used,  perhaps  because  of  the  number  of  POIs  in  the  picture,  the 
time  taken  for  the  algorithm  to  break  all  the  passwords  varied  widely. 

5.3  Algorithm  Difficulties  and  Solutions 

Our  results  were  not  directly  comparable  to  Zhao  et  al.’s  results  since  the  testing  and  train¬ 
ing  data  used  were  different,  however  we  were  able  to  create  an  algorithm  that  cracks  PGA 
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Figure  5.16.  CDF  Results  of  Picture  009899.jpg 


passwords.  Our  algorithm  used  a  significant  amount  of  memory,  hard  disk  space,  and  CPU 
time  to  sort  and  compare  the  many  coordinate  points  gathered  as  password  guesses,  as 
described  in  Section  4.3. 

Text-based  passwords  are  normally  stored  using  a  hash.  It  is  unknown  how  Microsoft 
stores  PGA  passwords  but  our  method  described  in  Chapter  2  used  a  significant  amount  of 
storage.  Python2  dictionary  and  list  data  structures  were  used  to  keep  track  of  passwords 
that  were  cracked  and  the  number  of  guesses  required  to  crack  each  password.  Suo  et  al. 
mentioned  that  memory  storage  for  password  guesses  is  a  difficult  problem  with  PGA  [2], 
In  addition  to  memory  problems,  the  CPU  was  not  powerful  enough  on  our  device  to  handle 
the  amount  of  work  necessary  to  run  the  algorithm. 

To  address  the  memory  issues  and  the  slow  execution  on  our  architecture,  we  used  Amazon 
Web  Services  (AWS)3  to  run  the  algorithms.  We  created  an  instance  of  a  c4.xlarge  Ubuntu 
server  with  16  GB  of  memory  and  4  CPUs.  Due  to  cost  factors,  the  time  spent  using  the 
AWS  instance  was  kept  to  a  minimum,  roughly  $45.  The  algorithm  was  run  for  each  of  the 
15  pictures,  on  separate  CPUs  for  efficiency. 

Even  with  AWS,  however,  we  were  unable  to  find  results  for  Figures  3.1(k)  and  3.1(1) 
for  which  the  program  failed  and  never  completed.  There  was  no  error  message,  such  as 
“MemoryError,”  to  indicate  what  caused  the  failures.  Attempts  to  display  an  exit  status 


2https://www.python.org/ 

3https://aws. amazon.com/ 
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in  the  terminal  (i.e.,  “echo  $?”)  also  failed.  The  same  results  were  found  after  running 
the  program  multiple  times  for  each  of  those  pictures.  We  assume  there  was  possibly  an 
excessive  number  of  passwords  generated  for  these  pictures.  Perhaps  there  were  far  more 
POIs  for  these  than  for  the  other  successful  pictures.  Fortunately,  we  achieved  results  for 
the  latter  pictures. 
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CHAPTER  6: 

Conclusions  and  Future  Work 


In  this  chapter,  we  will  discuss  the  accomplishments  of  this  thesis,  our  recommendations 
to  improve  the  security  of  PGA,  and  future  work  that  can  be  done  to  continue  the  research. 

6.1  Conclusions 

Each  picture  from  the  Arizona-Turk  study  was  investigated  in  this  thesis  for  its  strength  as 
a  background  picture  for  PGA.  It  was  found  that  strong  background  pictures  have  a  wide 
variety  of  POIs.  More  POIs  in  a  picture  implies  that  there  are  many  more  gestures  a  user 
can  choose  from  in  creating  a  password.  It  is  assumed  that  users  will  choose  from  among 
the  POIs  to  assist  their  choice  of  password  gestures. 

An  important  benefit  of  this  thesis  is  the  creation  of  a  program  that  can  crack  gesture  pass¬ 
words.  We  provided  a  description  on  how  to  crack  passwords  for  PGA.  Using  data  given 
by  Zhao  et  al.,  we  created  visual  representations  demonstrating  the  POIs  and  passwords 
of  the  pictures  for  the  Arizona-Turk  study.  Visuals  were  created  to  show  efficiency  of  the 
program  we  designed  to  offer  supplementary  resources  to  understand  the  limits  of  security 
of  PGA. 

Strength  requirements  for  PGA  passwords,  just  as  there  are  for  text-based  passwords,  will 
improve  the  security  of  PGA.  For  example,  strength  requirements  for  Windows  8  and  Win¬ 
dows  10  might  be  to  increase  the  number  of  gestures  per  password,  add  new  types  of 
gestures,  and  ensure  the  picture  chosen  by  the  user  contains  numerous  POIs  dispersed 
across  the  picture.  Using  a  smaller  error  distance,  as  discussed  in  Section  2.1,  will  force 
an  attacker  to  make  more  guesses,  however  this  can  cause  false  negatives  when  valid  users 
attempt  to  log  in.  Until  such  strength  requirements  are  available,  we  conclude  that  it  would 
be  beneficial  to  use  a  different  means  of  authentication  for  the  security  of  government  in¬ 
formation. 
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6.2  Future  Work 

We  have  developed  a  working  program  that  produces  sensible  guesses  to  crack  PGA  pass¬ 
words.  Ideally,  this  program  can  be  improved  in  the  following  ways: 

•  Most  importantly,  the  algorithm  can  be  enhanced  by  making  fewer  guesses. 

•  Advancements  can  also  be  accomplished  by  refining  memory  issues  and  increasing 
speed.  This  can  be  done  by  using  a  better  POI  detection  program,  and  considering 
programming  languages  other  than  Python. 

•  Since  the  coordinate  points  guessed  in  the  algorithm  are  made  in  order  from  the 
bottom-left  to  the  top-right,  an  improvement  might  be  to  randomize  the  order  of 
password  guesses  in  the  list  of  guesses  made  for  each  heuristically  ordered  pattern, 
as  described  in  Section  4.3. 

•  Another  solution  to  the  same  problem  may  be  to  begin  at  the  center  of  each  POI, 
which  would  “hit”  the  commonly  used  midpoints  of  the  circle. 

•  Furthermore,  the  algorithm  can  be  designed  to  construct  password  guesses  outside  of 
the  POIs  in  the  picture,  but  at  that  point,  it  would  be  brute-forcing. 

•  Finally,  it  is  intended  that  the  program  works  for  unseen  pictures.  This  may  be  uti¬ 
lized  by  adding  an  algorithm  that  locates  POIs  and  records  the  coordinate  locations 
of  the  POIs.  With  this  information,  the  brute-force  algorithm  in  Chapter  4  can  guess 
passwords  for  unseen  pictures. 
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